GAPFormer: Geometry-Adaptive Propagated Transformer for Point Cloud Representation

Wanlu Zheng^1,2   Shaorong Wang^1,3

¹ School of Information Science and Technology, Beijing Forestry University, Beijing, China
² School of Software and Microelectronic, Peking University, Beijing, China
³ Engineering Research Center for Forestry-oriented Intelligent Information Processing of National Forestry and Grassland Administration, Beijing, China

TL;DRPropose a geometry-adaptive Transformer for 3D point cloud semantic segmentation that integrates spherical harmonic geometric encoding, dynamic positional bias, and point-wise convolution attention

GAPFormer addresses the limited geometric modeling capabilities of Transformer-based methods in 3D point cloud segmentation. It proposes three key modules: SHGFM for direction-aware geometric encoding via spherical harmonics, Geo-DPB for geometry-aware dynamic positional bias, and PCBA for multi-level channel-spatial feature fusion. The framework achieves state-of-the-art performance on ScanNet v2, S3DIS, and ScanNet200 benchmarks.

Enhance Transformer-based point cloud segmentation by explicitly injecting geometric priors at multiple levels: (1) spherical harmonic basis functions for direction-aware feature encoding, (2) geometry-guided dynamic positional bias for attention enhancement, and (3) point-wise convolution-based channel-spatial co-optimization for feature refinement.

Fig. 1. GAPFormer Network Architecture.

The overall architecture follows an encoder-decoder paradigm: raw point clouds are transformed into high-dimensional features via sparse convolution, then a hierarchical Transformer encoder progressively learns multi-scale geometric features with cascaded pooling, while the decoder upsamples to reconstruct fine spatial details for accurate segmentation.

Fig. 2. SHGFM Module.

SHGFM computes direction vectors, magnitudes, and normalized vectors for each neighbor, then derives polar/azimuthal angles to calculate spherical harmonic basis functions (degree 0-2). These SH coefficients are concatenated with original features and relative distances, then fused through a residual MLP to produce geometry-prior-enhanced Key-Value pairs.

Fig. 3. Geo-DPB Module.

Geo-DPB precomputes all possible 7D geometric feature vectors within a local 3D neighborhood, maps them through a 3-layer MLP (ReLU+GeLU activations) to generate positional biases stored in a hash dictionary for efficient inference. During attention, biases are retrieved via quantized offsets and additively injected into attention weights.

Fig. 4. PCBA Module.

GAPFormer achieves consistent improvements across all three benchmarks. On S3DIS, it notably improves per-class mIoU for board (+5.4%), bookcase (+2.9%), and sofa (+2.1%) over PTv3, demonstrating superior fine-grained structure segmentation. Ablation study confirms that all three modules contribute synergistically, with the full model achieving the best mIoU of 72.6%.

Qualitative Results

Fig. 5. Qualitative comparison of S3DIS segmentation results.

Explicitly encoding geometric priors through spherical harmonics and dynamic positional biases significantly enhances Transformer-based point cloud segmentation. The key insight is that direction-aware frequency-domain encoding (SHGFM) combined with geometry-guided attention bias (Geo-DPB) addresses the fundamental limitation of conventional coordinate embedding approaches.